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ABSTRACT 

- This meta-analysis explored how measuring progress 
toward long- versus short-term goals relates to contrasting outcome 
measures of student achievement. Twenty-one controlled studies, that 
provided sufficient data for the calculation of effect size, were 
coded in terms of measuremerc iiethod (toward long- versus short-term 
goals) and type of achievement outcome (probe-like versus global 
achievement test). Analogues to analysis of variance conducted on 
weighted unbiased effect sizes (UES's) indicated an interaction: when 
progress was measured toward long-term goals, UlS's on global 
measures were higher than on probe-like outcomes; when progress was 
measured toward series of short-term goals, the reverse was true. As 
demonstrated in this meta-analysis, short-term goal measurement may 
be misleading, students may master a series of instructional 
objectives, even though progress may be limited on more global 
indices of achievement which better represent the true desired 
outcome performance. Consequently, for special education teachers who 
monitor mastery of short-term objectives, caution may b^ in order: 
curriculum-based assessment of long-term goals may represent a 
necessary supplementary strategy for validly assessing pupil 
progress. References, tables, figures, and a list of reports included 
in the meta-analysis are appended. (Author/PN) 
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Abstract 



Thit ntta-tnalrtit txplortd hoM mtaturing ttudtnt progrttt toward long- 
vt. thort*tfri« goals afftctt achifVMtnt outcontt. Tutnty-ont 
control U ' ^litt wtrt codtd in Umt of mtaturMtnt mtthod (toward 
long* vt. t^ i^'ttni goalo) and typt oi achitvfmtnt outcomt (probt-likt 
vt. global achitvtntnt tttt). Analogutt to analysis of variaiict 
conducttd on wtiglittd unbiattd tfftct tixtt (UE8t) indicattd an 
inttractiont Uhtn progrttt wat mtaturtd toward long-ttrm goaltf UESt on 
global mtaturtt wf rt liiglitr titan on probt-likt outcomtti wlitn progrttt 
wat mtaturtd toward ttritt of tliort-ttrm goalt, tlit rtvtrtt wat trut^ 
Impi tcationt for tptcial tc'Mcation practict art ditcutttd. 
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Effects of Long- and Short-Term Goal Assessment on Student Achievement 

In iptciil tducationi commtrciil norm-rtftrtncfd ichitvtmtnt ttttt art 
the traditional <Tinda1 tt iK, 198S) and continutt to rtprtttnt a prtvalant 
(6oh, TttloM, & Fuller, 1981) meiturement tool for generating individual iied 
educational progr«iit and for evaluating the effects of those prograns. 
Nevertheless, it has been criticiied increasingly <see Tindal ei a1«, 198Si 
Ysseldxke & Thurlowy 1984). With respect to generating educational progrms, 
critics contend that the abilities measured by these instruments frequently lack 
necessary conceptualization (Yss^ldykSi 1979), and relatedly that the tests often 
fail to demonstrate adequate psychometric properties <8a1via ti YsseldyKe, 198S). 
In terms of program evaluation, critics argue that these measures fail tot (a) 
indicate the extent to which specific educational objectives have been attained 
(Skager, 1971), (b) provide enough alternate forms to permit ongoing progress 
monitoring, (c) sample the domains of interest comprehensively (Zigmond & 
Silverman, 1984), and (d) relate to curricular materials (Armbruster, Stevens, k 
Rosenshine, 1977) Jenkins & Pany, 1978). 

In response to these problems, ongoing cr i terion-ref trenced, 
curricu1um*based assessment (CBA) strategies ^ave been developed. Uith CBA, 
measurement pr cedures are designed to match students' program objectives. 
Alternate test forms are drawn directly from curricula specified i^t objectives 
and are adhiinistered at regular intervals during intervention) student progress 
data are evaluated regularly with reference to the performance criteria specified 
in objectives) and individualized programs are tested formatively and modified 
over time as required to insure effective instructional programs and attainment 
of objectives. Therefore, with CBA, instructional program evaluation it ongoing 
and based in the curriculum) program development is ind*jctive, in response to the 
ongoing program evaluation data. 
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Rttttrch indicatft thtt such ongoing CBA of studtntt^ ittilmitnt of goals 
and objtctlvts rtprtstnts an tfftctivt alttrnativt approach to program 
dfvflopntnt and tvaluation. In a quantltativt synthtsis of availabit controlltd 
studltSi thf avtragt tfftct sizt was .70 (Fuchs It FuchSy in prtss). This 
indicatts thaty in ttrns of tht standard normal curvt and an achitvtmtnt ttst 
scalf with a population mtan of 100 and standard dtviation of ISy tht ust of CBA 
to dtvtlop and tvaluatt instructional programs ovtr timt can bt txptcUd to raist 
tht typical achitvtmtnt outcomt scort frfim 100.00 to 110.90| or from tht 90th to 
tht 76th ptrctntilt. 

Additionally! tht rtquirtmtnts of ftdtral Itgislation sttm to indicatt 
tht importanct of CBAi Tht lEP mandatt of PL 94-142 rtquirts sptcial tducators 
to sptcify long-ttrm goalsi short-ttrm obJtctivtSy and tvaluation proctdurts for 
monitoring studtnts^ attainmtnt of obj^ctivts. Assuming that tht inttnt of this 
Itgiflation was to tncouragt compatibility and congrutnct bttwttn 
goals/objtctivts and pupils^ curricula, thtn tht lEP mandatt rtquirts a CBA 
approach to p?ogrtss tvaluation. 

Dtspitt tht appartnt tfftct ivtntss of and sttming ntctssity for CBA, it 
rtmatns uncltar how practitiontrs should dtsign CBA proctdurts to monitor 
studtnts^ attainmtnt of goalf and objtctivts and how alttrnativt practicts rtlatt 
to studtnt achitvtmtnt outcomts. Currtntlyy practitiontrs can st!tCi: bttwttn two 
typts of CBA| ont focusing on tht attainmtnt of long-ttrm goals (CBA*goa1) and 
tht othtr of short-ttrm objtctivts (CBA-obJtctivt) • 

Uith tht CBA-goal approach, an annual curr iculum-bastd goal is sptcif ltd 
and a largt pool of rtlattd mtasurtmtnt ittms is crtattd. From this mtasurtmtnt 
pool, substts of ittms, or monitoring probts, art drawn randomly (stt Fuchs, 
Dtno, 4c Mirkin, 1964). Tht difficulty Itvtl of tht monitoring probt rtmains 
constant ovtr a long timt. Contrastingly, with tht CBA-obJtctivt approach, a 
strits of objtctivts corrtsponding to sttps within a hitrarchical curriculum is 
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Curr icu1uii*BiMd Attttimtnt-S 
tptcifitdi tnd t ttritt oi rtUtivtly circuntcribtd, uitll poolt of ittnt trt 
crtatfd, tich of which corrttpondt to a tptcif ic obJtcti*^t <ttt Lindtlty, 1971 1 
Uhitt & Hiring, 1980). Tht difficult/ Itvtl of mtttriil on which itudtntt irt 
meiturtd incrtittt it ttudtntt nitttr tht fttqutntii11y*rt1ittd objtcttvtt. 

Both typtt of CBA irt ongoingi cri ttr ion-rtftrtnctdf curr iculun^bittd, 
ind tnjoy strong curriculir vilidity or corrttpondtnc* bttwttn ttitt ind 
progriimitic goilt ind objtctivtt (McClung cittd in YiIom It Pophmy 1983). 
HoMtvtr, thttt trsttfiit do difftr conctptuil 1y« CBA*obJtctivt ipptirs to hivt 
ttrongtr inttructionil vilidity or corrttpondtnct bttwttD ttttfi ind intiruction 
(Yilow It Pophinii 1983). Tht monitoring probtt for thort*ttrm litMurtmtnt irt 
rtlittd dirtctly to currtnt inttructionil mittriili to, for txmpit, if in 
inttructionil inttrvtntion it introduction of tht r*contro11td phonict rult, tht 
monitoring mtiturt it rtiding r*contro11td wordt. AUtrnittlyi with CBA*goi1 , 
tht monitoring probtt irt not rtlittd to tht inttructionil mittriil. Tht 
inttructionil inttrvtntion miy bt introduction of thi r*contro11td phonict rult, 
whtrtit tht monitoring mtiturt miy involvt oril rtiding flutncy, iccuricy, and/or 
comprthtntion on ttcond gridt pittigtt. 

Although CBA-objtctivt miy tnJoy ttrongtr inttructionil uilidity, 
CBA-goal it idvintigtout in othtr rttptctt. It pottttttt bttttr conttnt vilidity 
or rtprtttntition of tht ultimitt dttirtd ptrforminci. -t-f rtiding 
f lutncy/comprthtntion (Yilow & Pophim, 1983). Additionilly, itt concurrtnt 
vilidity or corrtlition with othtr mtiturtt of ichitvtmtnt ipptirt to bt ttrongtr 
thin thit of CBA-obJtctivt (Fucht, 1982). 

Tht tmtrgtnt qutttion, ind tht focut of tht currtnt mtti-inilytit, it how 
wtll thttt typtt of ongoing cr i ttr ion*-rtftrtnctd, curr iculum^bittd itttttmtnt 
ttrittgitt rtlitt to outcomt mtiturtt of ttudtnt ichitvtmtnt. Tht invtttigition 
of thit qutttion thould htip prictitiontrt ittttt tht rtlitivt mtritt of tht two 
typtt of CBA ind ttltct CBA monitoring proctdurtt. 
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Curr iculum-Btstd AMftfmtnt*-^ 
Mtthod 

Starch Proctdurt 

Thf fttirch for ptrtin^^nt ttudiit to includt in tht mt tt-tntlxsls 
conprlstd four tttpt. First, tmploxing tht Thf aurum of PmycholoQictI Indtx 
Ttrrom (APA, 1982), multiplt dtmcriptors wtrt gtntrittd for Kty ttms. For 
txamplty itudtnt achi tvtwtf^t ilttrnittlx wit rtprtstnttd by 'studtnt progrtsftf' 
*goa1 attainmtnt/ and 'tducitionil tfftctt.' Stcond, thtst t«nit facilitittd i 
computtr starch of thrtt on-1in# data baatti <a) ERIC» a data bast of tducational 
mattriaU from tht Educational Rtsourctft Information Ctnttr consisting of 
abstracts from Rtttarch in Education and Currant Indtx to Journals in Educationt 
(b> Comprthtntivt Disttrtation Ahatraetai and <c) Paycholoolcal Ahmtracta, 
Third, tmploxing similar Ktx dtscriptors, a manual starch ^as conducttd of fivt 
tducaticnal Journals for tht xt^rs 1973 through 1983. Thtst Journals wtrti 
Aretrican Educational Kaaaarch Journal. Journal of Laarnino Disabil i tits. 
Journal of Prtcimion Ttachino, Journal of Soteial Education. Md LtlCOliifl 
Dltability Quarttrlx. Fourth, tht rtftrtnct stctions of rtltvant paptrs along 
with idtntifitd b'bl iographits wert txplortd for additional studits. 

Crittria for Rtltvant Studits 

A studx was considtrtd for inclusion if it tmploxtd a control group to 
tvaluatt tht tfftcts of curr iculum-bastd monitoring on acadtmic achitvtmtnt. Such 
monitoring was dtfintd as curricu1um*-bastfd data colltction that occurrtd at Itast 
twict wttklx, with dtcisions conctrning tht adtquacx of programs formulattd on an 
individual, not group, basis. Studits wtrt txcludtd that (a) monitortd social 
bthaviors, (b) primarilx focustd on tht ust of bthavior modification, whilt 
tmploxing tim^ strlts to ttst txptrimtntal tfftcts, (c) providtd ttst fttdback 
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Curr icu1um*Bittd Attttmtnt'-? 
only to ttudtntt, and/or (d) tnploytd colltgt tgt tubjtctt. <Othtr fictort, tuch 
at Inttrufitntation, mtthodologicil rigor, and adequacy of dtcitionmaKing mtrt 
codtd at variabitt, and rttuitt rtlattd to thttt variabitt art rtporttd tIttMhtrt 
[f.g., Fucht & Fuchti in prtttl.) 

Tht ttarch yitldtd 29 ttuditt that ntt tht crittria tttablithtd for 
Inclutlon. From thttt ttuditt, 11 wtrt tliminattd btcautt of intufficitnt data 
for calculating tfftct tizt. 

Data Extracttd Studitt 

Calculation of, rtlattd attuaotionm about, and intaroratat ion of tfftct 
tizt, Rttuitt of tht ttuditt wart trantfomtd to tttimattt of tfftct tizt, 
typically calculattd by tubtracting tht trtatmtnt ntant and dividing by tht 
control group ttandard dtviation. For ttuditt reporting rtltvant mtant and 
ttandard dtviationt for both groupt, tfftct tiztt wtrt calculattd from thttt 
litaturtmtntt. For ttuditt not rtporting mtant and ttandard dtviationt, tfftct 
tiztt wtrt calculattd fron othtr ttatittict, tuch at £ or £ valuta (ttt Blatt, 
McGaw, & Smith, 1981). Uhtn prttttt difftrtnctt or analytit A covarianct wtrt 
rtporttd, alttrnativt proctdurtt for calculating tfftct tizt wtrt uttd, at 
pottiblt, to control for thott difftrtnctt (ttt Glatt tt al., 1981). For 
purpottt of analytit, an tfftct wat givtn a potitivt tign if tubjtctt achitvtd 
grtattr tcortt in tht tytttmatic monitoring trtatmtnt. 

Sinct thit tttimator of tfftct tizt it biattd potitivtly, ttptcially for 
tmall til. tach tfftct tizt wat convtrttd to an unbiattd tfftct tizt (UES) by 
multiplying tht tttimattd tfftct tizt by a corrtction factor (ttt Htdgtt, 1981). 
Thit proctdurt corrtctt for incontitttncy in tttimating trut from obttrvtd tfftct 
tiztt (Htdgtt, 1981). Tht difftrtnct bttwttn tht obttrvtd and unbiattd tfftct 
tiztt wat ntgliblt (X ■ .019, SD ■ .025) at hat bttn dtmonttrattd tlttwi.trt 
(Bangtrt-Drownt, KuliK, & Kulik, 1983). Ntvtrthtlttt, UESt wtrt tmploytd to 
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inturt thf nathcniatical tractablllty of tht datit* 

Thf ttetlttlcal proptrtltt of tfftct tlit dtptnd on tht modtl for tht 
obstrvationt In tht txptrlmtnt. In thit mt ta*ana1ytlt, It wat attumtd that 
obttrvationt art dittrlbuttd Indtptndtntly normally withhi groupt of tht 
•xptrlmtnt. Tht rtUttd Inttrprttat ion of tht population tfftct tizt \% that It 
rtprtfttntt tht mtan difftrtnct ont would obtain If tht dtptndtnt variabit Mtrt 
»ca1td to havt unit varianct within groupt of tht txptrlmtnt. Thutf tht tfftct 
tizt it tht mtan difftrtn.t rttxprttttd In unit tcaltd to that^T^l to rtmo^t tht 
dtptndtnct of tht arbitrary tcalt factor <f. Uhtn obttrvationt in tht 
txptrimtntal and control groupt art dittrlbuttd normallXy tfftct tU. can bt uttd 
to quantify tht dtgrtt of ovtrlap bttwtn tht dittrlbtlont of obttrvationt in tht 
txptrimtntal and control groupt. Btcautt thit tfftct tizt it tht ttandardiztd 
scort of tht txptrimtntal group mtan in tht control group dittr Ibutioni it 
rtprtttntt tht proportion of control group tcortt that art Ittt than tht %vtragt 
scort In tht txptrimtntal group (Htdgtt k Olkiny 1985). 

Thit paramttric point tttlmatt for tfftct tizt wat ttltcttd ovtr 
nonparimttr ic tttimatort btcautt nonparamttr Ic tttlmatort can bt computtd ont/ 
whtn raw data of tach ttudy art availabit and btcautt tuch tttimatort probably 
art Ittt tfflcltnt than paramttric counttrpartt whtn tht attumptiont of 
paramttric proctdurtt art tatitf ltd. Thut, Htdgtt and Oikin (1989) rtcomtnd 
that nonparamttr ic tttimatort bt uttd only whtr It It tutptcttd that paramttric 
assumptiont art violattd ttrioutly. 

Tht paramttric point tttlmatt for tfftct tizt alto wat ttltcttd ovtr 
tttimation of an tfftct magnitudt battd on tht idta of varianct accounttd for dut 
to tht Introduction of an txplanatory variabit, tuch at corrtlation cotfficitntt 
and ratiotf intra^clatt corrtlation cotff icitnttf and tht omtga^tquartd Indtx. 
Although tuch indlctt art intultlvtly apptalingp thty art not tuittd for 
combination acrott ttuditt (Htdgtt & Oiklny 198S)i Thty art nondirtct ional and 
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dtptnd on functions of trbitrary dttign d«cisiont| such it tht pirticulir 
dtfinition of groups or pitttrnt of X vilutt stitcttdi it Mtll it on tht 
undtrlying rtlition bttmtn thtortticil conttructt <Htdgtt Ir Olkini 1969). 

Efftct tilt iQQrtQ i ttQi|. Ouidtl intt vftrt tttiblithtd to tnturt thit tich 
rtltvint tfftct Mit counttd only o?ict in inilytttt Uhtn in tfftct Mit mtiturtd 
by ttttt thit fiiltd to rtprtttnt dimtntiont rtUvint to the mtti<-iniixtit <i.t.9 
Rtiding Ccmprthtntion ind Structuril Anilytit Subttttt of tht Stmford Diignottic 
Rtading Tttt), thttt rttuitt Mtrt pooltd. For txmplt, if ichitvtmtnt within a 
ttudy wtrt mtiturtd with thrtt globil ttttt ind two probt*1ikt measur^S) the 
thrtf tfftrt tiztt for tht global ttttt would bt iggrtgittd it would bt dont for 
tht two probt-likt ttttt. So two, rithtr thin fivt| tfftct tiztt would bt 
includtd for tuch i ttudy. 

Thtrt wtrt 96 tfftct tiztti with bttwttn 1 ind 12 tfftct tiztt ptr ttudy. 
Anilyttt indicittd no ttititticil dtptndtncx bttwttn tfftct tizt mignitudt ind 
numbtr of compiritont ptr ttudx < £».12 ). Thtrtforti UESt wtrt iggrtgittd it 
tht individuil tfftct tizt Itvtl. 

In combining UESt| i weighting proctdurt wit tmplo/td to iccount for tht 
fict thit tht viriinct of tht tttimitor dtptndt on timplt tizt| in which 
fttimittt from ttuditt with lirgtr timplt tiztt irt mort prtcitt thin thott from 
ttuditt with tmilltr timplt tiztt <Htdgtt & Oikin, 1969). Ktnct, tht wtighting 
proctdurt givtt wtight invtrttly proportionil to tht viriinct within tich ttudx: 
Uith 1 lirgtr H/ tmilltr viriinctf i lirgtr wtight it ittigntd. 

To combint UESt, i dirtct wtighttd lintir combinition of tttiro^tort 
proctdurt wit tmploxtd btciutt Htdgtt ind Oikin <1989) hivt dtmonttrittd thit 
tuch 1 mtthod it compiribit to, but timpltr and mort intuitivtlx ipptiling thin, 
ilttrnitivt proctdurtt. In tuch in iggrtgition, lirgt timplt ttititticil thtorx 
for tttimiting tfftct tizt from i ttritt of ttuditt it tmploxtd, ind Htdgtt ind 
Oikin <1989) dtmonttrittd thit thtortticil oritntition it rtitoniblx iccuritt 
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Curricu1uii**Biitd Aiitiinitnt**10 
whtn tfftct ii2ti trt Ittt than 1 ,5 in tbtolutt magnitudt and tmplt tizti art at 
Itatt 10. Thtft conditiont Mtrt ntt in tht currtnt ii»ta*ana1yiii. Ntvtrthtlttt , 
tuch an aggrtgation alto attuniti ptrftct lintar tquatability bttMttn dtptnfltnt 
litaturti and rtlattdly timilar optrationai Szationt oi tht conitru^ti ntaiurtd. 
Thtft atiumptioni nay bt violattd through noncmparabil ityi ntasurtmtnt trroPi 
and prtitnct oi uniqut factors or invalidity (Htdgts & Oikin, 19Pn). Thtrtfort, 
rtiultiiBUit bt contidtrtd within tht confintt oi thttt pottntial statistical 
probi tmt J 

Study ftaturti. To dttcribt study ftaturtt ptrtintnt to tht currtnt 
invtitigationi tmo major tubttantivt variablti wtrt idtntifitd and codtd for tach 
ttudy. Tht f irtt ttudy ftaturt Mat typt of coal . This variabit had two ItvtU 
that difftrtfit iattd ttuditt in which progrtii toward 1ong*ttrni goals (CBA*-goa1) 
was noritortd froTi ttuditt in which progrttt toward a thort*-ttrni objtctivt or a 
ttritt of thort-ttrm objtctivtt (CBA-obJtct i vt) wat monitortd. 

Studitt in which progrttt toward iong**ttrni goal wat nonitortd involvtd 
tht tptcif ication of a Itvtl of nattria! on which a ttudtnt wtt txptcttd to bt 
prof icitnt within tht ntxt 19 or nort wttkt. For txanplti for a ttudtnt 
currtntly rtading proficiently on printr mattriali a ttudtnt't goal might tptcify 
that, in 29 wttkt, a ttudtnt would rtad 79 words ptr minutt corrtct with 90X 
accuracy on ttcond gradt rtading pattagtt. Thtni for tht ntxt 25 wttkt, 
mtaturtmtnt probtt would bt tampltd randomly from ttcond gradt rtading pattagtt, 
rtprtttnting approximattly tquiualtnt tampitt of mtaturtmtnt mattrial. 

Studitt in which progrttt toward thort**ttrm goalt wat monitortd rtquirtd 
tht identification of a ttqutnct of tmall ttgmtntt in a hitrarchical curriculum 
to bt matttrtd by tht ttudtnt. For txampit, tht ttritt of objtctiuts might 
tptcify that tht ttudtnt would rtad, with 90^. accuracy, flathcardt firtt with 
contonant*uowt1*contonant wordt, ttcond wi^h final t wordt, and third with doubit 
vowtl wordt. Procttding in a fathion paralltl to tht tptcif ication of 
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Curricu1uii*Btitd Attttmtnt*U 
objtc^lvtt, nttturtmtnt probtt first mould b» dr«Mn from flithcirdt nith 
contonint*voMt1*contontnt Mordt until tht uttttry crittrion Mit ^chitvtd by tht 
fttudtnt on thit domiin. Thtn, tht mtiturtiitnt domiin mould chingt to that probtt 
wtrt flifthcirdt inith final t wcrdt, ind to on. 

Tilt ttcond ttudx ftiiurt wit outconit nf^tur^. Thit variabit alto had two 
Itvtit: dtptndtnt mtaturtt timilar to tht monitoring probtt and mort global 
achitvtmtnt ttttw. Employing t' ' txanpitt providtd abovt, probt*1ikt outcomt 
indictt wtrt oral rtadinc ratt on ttcond gradt pattagtt or ptrctntagt rtad 
corrtctlx from flathcardt with final t wordt] global achitvtmtnt ttttt wtrt tht 
Structural Ar^alytit and Rtading Comprthtntion Subttttt of tht Stanford Diagnottic 
Rtading Tttt. 

In addition to thttt two tubttantivt ftaturtty a third, mtthodological 
variabit wat codtd for tach ttudx, duration of tht 'crtatmtnt. Thit variablt had 
titrtt Itvtlts trtatmtntt impltmtnttd for Ittt than 3 witkt <codtd '1')] 
trtatmtntt latting bttwttn 3 and 10 wttkt <codtd '2')] and trtatmtntt continutd 
for mort than 10 wttkt <codtd '3'). 

Two rattrt indtptndtntlx codtd 10 of tht 18 ituditt <S6X). Ptrctntagt of 

2 

agrttmtnt for tht rattrt on txpt of goal wat 77 and 83 whtn progrttt toward 
long- and thort-ttrm goalt wat tht rttptctivt Itvtl of tht variabit, with a mtan 
inttrcodtr agrttmtnt of 60. For outcomt miaturt, tht ptrctntagtt of agrttmtnt 
for probtt and global achitvtmtnt ttttt wtrt ?4 and 86, rttptctivtlx, with a mtan 
ptrctntagt of 90. Ptrctntagt of agrttmtnt for duration of trtatmtnt wat 100 for 
all Itvtit of tht variablt. 

A prtviout invtttigation <Fucht & Fucht, in prttt) txplortd 
rotthodological qualitx of tht ttuditt and idtntifitd no rtlation bttwttn tfftct 
tizt magnitude and ttudx qualitx* AdditionallXi thit prtviout ttudx rtporttd an 
ovtrill tfftct tizt at wtll at tht rtlattd fail-taft tit Thutt, thttt rttuitt art 
not rtptattd htrt. 
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^ I 

04 tht 20 rtftrtncft liftttd in tht Apptndix, Mhich rtprtttnt 18 ttpArAtt ^ 
invfttigationtp' thtrt art 4 ditttrUtiontp U UApublifhtd ttudUt, and 5 Journal 
articltt. Among tht publithtd ptptrt, 2 apptartd (a Exctftilonal Chi^drtn. 2 in 
Amtricat. Educatmnal Rtmtareh Journi , ^nd 1 In ftitrlCin Jpymil ftf fitntil 
Df^icltncy, A total o4 3665 ftubjtctt participattd iA Ihttt otydltt, with 839C of %^ 
tht invtttigationt ttaploylng handicapptd subjttts. Of thttt handtcapptd pupilt, --^f 
93/. Mtrt mildly to modtrattly handicapptd and TA %itrt ttvtrtly handicapptd^ Tht J 
gradt Itvtl of thttt tubjtctt rangtd from prttchool through high tchool p with a 
mtdian gradt Itvtl of 3.6. Among tht 18 Invtttigationt, 8 \-)4%) fccuttd toltly 
on '%ht acadtmic arta of rtadinr^p 3 ilTA) on rtading and math, 2 (11%) only on 
math, and 1 <^) tach on (a) high tchool conttnt artat, (b> prttchool tkilU, (c) 
tptlling, <d> rtad'ng and tptlling, and <t) rtading, math, and tptlling. 

Rt tuitt 

Of tht 96 tfftct tiitt, 27 rtlattd to 1ong*ttrm goal mtaturtmtnt and 69 
to thort*ttrm goal mtaturtmtnt. Of tht 27 1ong*ttrm goal tfftct tiitt, 14 Mtrt 
attociattd with probt*1ij(t and 13 with global outcomt mtaturtt. Of tht 69 
thort-ttrm goal tfftct tiitt, 37 utrt rtlattd to probt^likt and 32 to global 
outcomt mtasurtt. 

RtUtion bttinttn trtatmtnt duration and othtr tfftct ttzt 4taturtm. A 
pair of X ttttt Mat run to dtttrmint nhtthtr mtaturtmtnt goal or outcomt mtaturt 
was rtlattd to tht duration of trtatmtnt. Thttt ttttt indicattd no ttattttically 
tignif leant attociat iont. For tht 1ong*ttrm goal tfftct tiitt, tht mtan codtd 
Itvtl of trtatmtnt duration (ttt abovt) wat 2.92 <8D « .27) f for tht thort*ttrm 
goal tfftct tiitt, 2.75 (8D ^ .54), 1 (95) « 1.52, flju Tht avtragt Itvtl of 
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trtatntnt duration for tfltct tiztt attocitttd with probt-liKt and global outcoRt 
ntaturtt, rttptct ivtly, wtrt 2.78 <SD « .91) and 2.76 <80 « .23), i <9S) ■ .24, 
Ail. 

RtUtion bttwttn maonituiO and ^»«tur>» oi af^>et TabU 1 

ditplayt tht wtighttd UE8t by <a) tht typt oi goal factor (1ong-ttr« goal va. 
thort-ttrm objtctivt^ and <b) tht outcont ataturt factor <proba^1iKt v«. global 
achitvMitnt tttt). To txaoiint tht rtlatioo btlwttn thttt viriablta and tfftct 
tizt magnitudt, Htdgtt'a <1984) chi-sqyart analogat to analytic, of ^iffanct i«at 
tmploytdt Uhtn convtntiontl «ni1>^!iift of virl4nc# it condiicttd 9* «fltct ffxtfi 
probltmt fxitt btcautt of tht pottibility that tytttMtic vartanct nill bt pooltd 
into tht tttinatt of error varianct. Mortovtr, violation of tht hovotctdatticity 
attumption it ttvtrt in rtttarch tynthttit, and thtrt It fittit rtatoli to btlitvt 
that tht Qtual robuttnttt of tht £ tttt Mill prtvail <ttt Htdgtti )984). Thut, 
Htdgtt't chi-tquart analogut Mat tmploytd to avoid thttt conctptual and 
ttatittlcal probltmt. 

At indicattd in Tabit 1, typt of goal wat not rtlattd to UE8, but outcont 
n»taturt productd a ttatittical 1y tignificant difftrtncty with tht mtan tfftct 
«izt of probt-likt mtaturtt .U of a ttandard dtviation hightr than that of 
global mtaturtt. Ntvtrthtlttt, ttttt for tht homogtntlty of tfftct tizt <Htdgtt, 
1984) indicattd that nont of tht four poolt of UESt rtprtttnttd a homogtntout 
ttti Statittical valuta for long- and thort-ttrm goalt and for probt-Hkt and 
global mtaturtt, rttptct ivtly, wtrt X^<2A> « 208.37, A^<M) « X^(44) « 

659.20, andXHSO) « 3S7.S4. Thtrtfort, additional analyttt Mtrt conducttd. 



Inttrt Tabit 1 about htrt 



Thttt additional analyttt addrttttd tht tfftct of typt of outcomt 
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liftturt Mithin ttch of tht txpt pi goti conditionty tnd tuggttttd tht prttonct of 
tn Inttrtction. At dttcribtd in Ttbit 2 tnd illuttrtttd in Figurt 1, within tht 
typt of goti conditiont, thtrt Kttrt ttttittictllx tignifictnt difftrtiictt bttwttn 
UE8t tttocitttd with tht probt-likt tnd tht globtl outcomt nttturtt. With 
CBA-obJtctivtp UE8t tttocitttd with probt-likt outcott vttturtt wtrt hightr thtn 
thott of globtl nttturtt. For C6A-got1 , tht rtvtrtt wtt truti UESt tttocitttd 
with globtl Mtturtt wtrt hightr thtn thott rtltttd to probt-likt outcomt 
littturtt. Sptcif icti Ixi within thort-ttnt goti twtturtiitnt, tht mttn tfftct tizt 
for probt-likt mttturtt wtt .40 hightr. tktn tfitt of ijlobtl nttturtt} within 
long-ttrm goti mttturtntnti tht tvtrtgt tfftct ti2t for probt-likt mttturtt wtt 
•51 lowtr thtn thtt of globtl nttturtt. 



Inttrt Ttbit 2 tnd Figurt 1 tbout htrt 



Ditcuttion 

Tht purpott of thit mttt-tntlytit wtt to invtttigttt how wtll Mtturing 
progrttt towtrd long- vt. thort-ttrm gotit rtltttt to contrttting outcomt 
mttturtt of ttudtnt tchitvtmtnt. Towtrd thit tnd, t littrttcrt tttrch wtt 
conducttd, rttulting in tht idtntif icttion of 18 rtltvtnt ttuditt thtt providtd 
mufficitnt informttion for tht ctlculttion of tfftct tizt. Thttt ttuditt wtrt 
codtd for long* vtu thort-ttrm goti mttturtmtnt tnd for probt-likt vt. globtl 
outcomt tchitvtmtnt mttturtt. To invtttigttt tht pottibilitx thtt thort- tnd 
long-ttrm goti r^ttturtmtnt or probt-likt tnd globtl tchitvtmtnt mttturtt night bt 
rtltttd to tht durttion of tht txptrimtntti trtttmtnt, ttudy durttion tito wtt 
codtd. Antlxttt indictttd no rtlitbit tttocittion bttwttn tithtr tubtttntivt 
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variabit and trtatntnt duration. 

Analogutt to analytit of varianct indicattd that tht nagnitudt of tfftct 
tiare wat not rtlattd to typt of goal on mhich monitoring occurred, but wat 
atiociattd with tht typt of outcomt mtaturt tmploytd, with a ntan difftrtnct in 
tfftct tizt of .11. Thit indicattt that in ttrmt of a standard normal curvt and 
aehitvtmtnt ttst tcalt with a population mtan of 100 and standard dtviation of 
IS, attttting outcomt with a probt-typt mtaturt can bt txptcttd to raitt tht 
typical achitvtfsant tcort attociattd with global mtaturet from 100.00 to 101.69. 
Such an tfftct alto indicattt that tht upptr 90% of tht dittribution of tfftct 
tiztt attociattd with outcomtt attttttd via probt-likt mtaturtt txcttdt 
apprcximattly 54X of tht dittribution for which tfftct tixt wat attttttd on 
global outcomtt mtaturtt. Thtrtfort, thit ttatittically tignificant difftrtnct 
apptart to rtprtttnt ont of minor practical tfftct. 

Additional analyttt tuggttttd a mort important tfftct, ont of 
inttractlon. Uhtn progrttt wat mtaturtd toward long-ttrm goalt, tfftct tixtt 
calculattd on global outcomt mtaturtt wtrt hightr than on probt*-1ikt outcomtt. 
On tht othtr hand, whtn progrttt wat mtaturtd toward a itritt of thort^-ttrm 
goalf, tfftct tiztt wtrt lowtr on global than on probt-likt outcomt mtaturtt. 

Thit finding may bt txplaintd in ttrmt of tht rtlativt ttrtngtht 
attociattd with ti)t difftrtnt goal mtaturtmtnt ttrattgis*. Long-ttrm goal 
mtaturtmtnt corrttpondt poorly with inttructional activititt, but comparati vtly 
wtll with global mtaturt» of rtading tkillt, including ttttt of dtcoding, word 
rtcognition, and comprthtntion <Dtno, Hirkin, Chiang, 1982] Fucht, 1981). On 
tht othtr hand, with short ^-ttrm goal monitoring, corrttpondtnct bttwtt.* 
inttruction and mtasurtmtnt is ont*-to*-onti howtvtr, as Quilling and Otto <1971) 
dtmonstrtttd, m^sttry of a hitrarchy of dtcoding skills rtlatts inconsisttntly to 
global achitvtmtnt indicts. 

Of courst in inttrprtting findings of this synthtsis, as with any 
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quantltitivt inttgrtgition, ont mutt linit gtntral izttiont to tituitiont tinilir 
to tht txptr liitntil/control trtitntnt ind dtptndtnt mtiturtt in tht primary 
rtttirch. Within tuch confinttp alttrnttivt txplinitiont for tindingt txitt. 
For txamplt, tomt mix inttrprtt rttuitt to tuggttt that, in ordtr to dtmonttratt 
ftptcial tducition iff tcti vtnttty prictitiontrt thould ttltct outcom mtaturt* 
that rtfltct tht txpt of goal monitoring thtx havt conducttd. HoMtvtri an 
alttrnativt and ptrhapt mort product I vt inttrprttation tuggtttt tht rtutrtti In 
ordtr to promott tht txpt of outcomt tptcial tducatort dttirt (i.t.p global 
growth vt. matttrx of ditcrttt curriculum unitt), goal monitoring mtthodt nttd lo 
bt ttltcttd cartfullx. 8ptclfica11X| at practi tiontrt dtvtlop thtir progrmmatic 
or lEP goalt and objtctivtt and rtlattd curr iculum-battd atttttmtnt proctdurtt 
for monitoring pupil progrttt toward thott goalt and objtctivttp both tht 
curricular and conttnt validitx of thtir mtaturtmtnt proctdurtt mutt bt 
addrttttd. Curricular validitx rtftrt to tht match btt'otn tttting and lEP goals 
and obJtctivtt{ conttnt vallditXi tht corrttpondtnct bttwttn tttting and tht trut 
domain in which prof icitncx it dttirtd CValow I Popham, 1983). Curricular and 
conttnt validitx art addrttttd timultantoutlx onix whtn practitiontrt writt 
"significant rather than trivial' lEP goalt and objtctivtt, which rtlatt wtll to 
tht trut dttirtd outcomt ptrformanct <Popham tt al.y 1989). Atttntion to thit 
dual crittrion allowt 'mtaturtmtnt*dr ivtn inttruction' <Popham tt a1., 1985) , or 
ongoing attttiiks^nt of pupil progrttt, to attumt an important tfftct on 
achitvtrotnt. It implitt that practitiontrt monitor progrttt toward 1ong*ttrm 
goalti an approach that apptart to promott a global tfftct on achitvtmtnt. 
Practitiontrt max with to utt thit ttrattgx to compltmtnt analxttt of thort*ttrm 
objtctivt matttrxi a txtttm that, on tht othtr hand, can guidt inttructional 
programming dtcitiont mort dirtctlx* 

Tht finding that 1ong*ttrm goal monitoring rtlattt bttttr to global 
achitvtmtnt outcomt mtaturtt max bt ttptciallx important in tht tducation of 
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handicYpptd ttudtntty who txpictllx htvt poorly dtvtloptd ttrittgitt for 
maintaining and trantftring tkillt (Andtrton-Innany Uaiktr, k Purctllp 1964t 
Uhitf, 1984). Short-ttm goal ntaturtntnt focuttt on inttructlonallx rtlaitd, 
rtUtivtly rf^tricttd donaint of liattrial for a ptriod of tint and thtnt upon 
matttrx of that mattrial, tht mtaturMtnt and inttructional focut f inul tantoutly 
changtt. Such a paradigw max bt probltmatic for at Itatt two rtatoiift. Firtt, it 
may ditcouragt ttachtrt from rtviMiing mattrial tuff icitntly to %Um for 
loiig-ttrm tkill mainttnanct. Stcond, a clott conntctlon bttiittn inttfuction and 
mtaturtmtnt max tncouragt ttachtrt to ortttnt fitw sKilU to ttudtntt Utthin tht 
framtwork of tht mtaturtmtnt iatk. For txamplt, if tht mtaturtmtnt proctdur* 
rtquirtt tht pupil to rtad contonant-vowtHcontonant wordt from a litt^ tht 
ttachtr max focut instruct ion on rtading contonant-'voMtl^coiitoiiant ifordt from^ a 
list. At nottd by Goodtttin (1982>9 thtrt max bt dangtr in txtng tht 
instructional format too clostly to tht asststmtnt dtvict or of narroMly dtfining 
conttnt-x-format domains of cr i ttrion^rtftrtnctd asstssmtnt. Such a rtstricttd 
instructional format may limit tht transftr of skills. Amort global, long-ttrm 
goal approach to mtaturtmtnt may tncouragt ttachtrt to incorporatt instructional 
proctdurtt that bttttr allow for tkill mainttnanct and gtntral ization. 

Ttachtrt may prtftr short-ttrm goal mtaturtmtnt btcaust it is tasitr to 
undtrttand and it guidtt instruction mort dirtctly by providing information about 
when to progrttt from ont skill to anothtr (Fuchs, Utsson, Tindal, Nirkin, & 
Dtnoy 1982). In fact, tvidtnct suggtsts tht prtdominant monitoring strattgy is 
asstttmtnt of thort-^ttrm objtctivt masttry through tht ptr iodic ust of conmtrcial 
cr ittr ion*-rtftrtnctd mtasui^^s such as basal strits masttry ttsts and tht Briganct 
(1978) Diagnostic Invtntory (Fuchs, Fuchs, AUarrtn, 1982). Ntvt^tht1tss, as 
dtmonstrattd in this mtta*ana!ysis, short-ttrm goal mtaturtmtnt may bt 
misltading. Studtnts nay masttr a strits of instructional objtctivts, dtspitt 
that progrtss may bt limited on mort global indicts of achitvtmtnt, which bttttr 
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rtprtfttnt tht trut dttirtd outcomt ptrformtnct. Confttqutntl/i for ttichtrft who 
monitor matttrx of thort-ttrn object ivtty caution nay bt in ordtri 
Curriculuii-btfttd ttfttftfttnt of long-ttm goals may rtprtttnt a ntctftftary 
ftuppltmtntiry ttrattgy for validly atfttiting pupil progrt^«. 
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Footnotft 



In iddition to thttt probltmt iitociittd wi th noncompiribi 1 ity, 
ntifturtnitnt trroTp ind tourctt of inviliditXi rtttirch dtsign fttturtt tnd 
multipit bthivioril ciuttt cin bt imong limiting fictort in iii^ttrprtt ing 
quantitativt inttgritiontt 

2 

Ptrctntagt of igrttmtnt wit cilculittd using tlit following fornKili 
(Coulttr cittd in Thonptonp Uhitt, k Morginp 1982)i Ptrctntagt of igrttiMnt « 
agrttmtntt bttwttn obttrvtr A & obttrvtr B/<igrftmtntt bt twttii A 4 B ♦ 
ditagrtrntntft bttwttn A & B ♦ onittlont by A ♦ omiiftiont by B). 

3 

Ont papt^ authortd by Haring <1971) and two additional rtportt by Haring 
tt Krug (197Sap 197Sb) dttcribtd atptctf of tht tamt invtfttigation. Thtrtfort, 
although It It rtportad that 18 ttuditt wtrt tmploytd in tht m«ta-ana1ytift, 20 
apptar In tht Apptndix dut to tht ttparatt lifting of tht Haring (1971) and tht 
Haring and Krug (197Sa, 197Sb) paptrs. 
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Tabit 1 



Utighttd Mtan UESti Valutt; and Chi*8quart Statittict at Afialogutt to 
Analxftift of Varianct by Typt of Ooal and Outcomt Htaturt Factors 



Factor 


Utighttd 1( 


1 Valut* 


a** 




Typt of 6o«1 






96 


.69 1 


Long-ttrm 




16.98 


27 




Short-ttrm 


.67 


24.82 


69 




Outcant Htaturt 






96 


6.63* 1 


Probt-1 ikt 


.72 


23.23 


49 




Global 


M 


19.06 


91 





"a tignficant 1 valut indicattt that tht wtighttd mtan it rtliably difftrtnt 



from zero. All 2. valutt art tigni-ficant btyond tht .001 ItMtl. 
**ti rtprtttntt numbtr oi UESt not numbtr oi ttuditt. 
* £<.05. 
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Ttbit ? 



Utightfd Mttn UESs, i Valutti and Chi*Squ«rt Stttitticft m Anitlogucft to 
Analyftift of Varianct for Probt*Likt and Global Outcomt Mtaturtt Mi thin 

Typt of Goal Conditions 



Txpt of Goal/ 












Outcomt Mtafturt 


Utighttd "9. 


1 Valut" 


u 






snort-Ttrm Goal 












Outcmt Mtaturt 






69 


94.78* 


1 


Probt-Likt 


.85 


22.97 


37 






Global 


.49 


11.94 


32 






Long-Ttrn Goal 












Outccnt Mtature 






27 


41.99* 


1 


Probe-Li kt 


.41 


7.32 


14 






Global 


.92 


16.73 


13 







" A significant i valut indicattt that tht wtighttd mtan it reliably different 



front zero. All lvalues are significant beyond the .001 probability level. 

b 

N represents number of UESs not number of studies. 

' £ < .001. 
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Figur* Caption 

f ioart 1. UnblMtd nran tfftct ftiztt (UE8t) for CBA-obJtetivt < > 

Md CM-go«1 ( — — > on probt-lik* and global outcoM Mavurtt. 
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